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ASYMPTOTIC  STABILITY,  IDENTIFICATION,  AND  THE  HORIZON  PROBLEM 

ABSTRACT:   Recent  literature  in  discrete  adaptive  control  has  emphasized  the 
importance  of  asymptotic  stability  of  the  adaptive  controller  in  obtaining 
convergence  of  system  parameter  estimates  to  their  true  values.   This  paper 
studies  the  relationship  between  these  results  and  the  problem  of  the  conver- 
gence of  first  period  decisions  in  planning  models  as  the  planning  horizon  time 
increases.   The  primary  results  to  date  have  been  based  on  stationary  and  pure- 
ly quadratic  cost  functions.   This  paper  extends  these  results  to  cost  func- 
tions containing  linear  terms  and  to  discounted  cost  functions.   The  main  re- 
sults are  a  set  of  sufficient  conditions  on  the  nature  of  cost  and  system  par- 
ameters under  which  first  period  decisions  converge  to  a  fixed  value  as  the  op- 
timization time  horizon  increases.   A  characterization  of  the  optimal  asymptotic 
controls  is  given  for  the  discounted  and  undiscounted  cases. 
I.   INTRODUCTION 

Consider  the  following  linear  system. 

(1.1)  x^^^  =  Ax^  +  Du^  +  v^ 

(1.2)  y^        ^^^^.^ 

In  (1.1),  X,  is  a  p-dimensional  column  vector  which  represents  the  state  of 

the  system  at  time  k;  A  is  a  pxp  transition  matrix;  D  is  a  pxr  control  matrix; 

u,  is  an  r-dimensional  control  vector;  and  v,  represents  a  random  disturbance. 

In  (1.2),  y  is  a  q-dimensional  vector  representing  an  observation  made  on 

the  system  at  time  k;  M  is  a  qxp  observation  matrix;  and  w  represents  noise. 

We  will  assume  that  v  ,v^,,..  and  w^,w„,...  are  independent  sequences  of  zero 

mean,  independent  and  identically  distributed  random  vectors  with  covariance 

matrices  V  and  W  respectively  and  that  x  is  independent  of  the  v.  s  and  w.  s 

^  ^  o        ^  1        J 

and  has  finite  covariance  matrix. 
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Linear  least  squares  prediction  and  filtering  may  be  done  for  the  system 

(1.1)-(1.2)  using  the  Kalman  Filter  [h],   which  yields  the  projections  x  , 

t  |k 
and  y^ij^  of  x^  and  y^  on  the  Hilbert  subspace  spanned  by  y^  .y^ , .  .  .  .y^^.   These 

projections  are  given  by 


(1.3) 


^|k  =  A   \|k 


^|k  =  "\|k'   •^"^' 

where  I  denotes  the  pxp  identity  matrix  and  x^|^=E[xJ.   The  weighting  matrix 
^^   in  (1.3)  is  determined  by 

(1.4a)     h^   =  Sj^M'[MSj^M'  +  W]"^,  k>l 

(1.4b)     S^  =  APj^,_^A'  +  V,  ^,^1 

(1.4c)     Pk  =  [I  -  A^MlSj^,  k^l 

where  +  denotes  pseudo- inverse,  '  denotes  transpose,  and  P^  is  the  covariance 

matrix  of  x  . 
o 

Now  consider  the  following  optimization  problem. 
(1.3)   Mi„  E,  .1^    l^p^_^^  .  ^-A,.-lVl  -  \.\   -  S..-1V1)  ' 
subject  to  (1.1), (1.2),  k-O.l N-1,  where  for  k.l,2,...,N,  Q,  ,  and  Q 

are  symmetric  positive  semi-definite  (psd)  matrices  and  G    ,  G      are  Ixp 
and  Ixr  row  vectors  respectively,  and  one  of  the  following  conditions  is  sat- 
isfied:  (a)  Cl2,k-1   ^^   positive  definite  (pd)  ;  (b)   Q^  ^  is  pd  and  rank(D)  = 
min(p,r).   Either  (a)  or  (b)  is  required  to  insure  the  existence  of  a  finite 
minimum  of  the  performance  criterion  and  to  insure  the  existence  of  matrix  in- 
verses for  the  dynamic  programming  solution  to  the  problem,  (1.5). 

In  an  extension  of  Gunckel  and  Franklin's  result  [  3  ]  for  the  pure  quad- 
ratic loss  function,  it  can  be  shown  (see  [  ?])  that  the  optimal  controls,  u^^, 
in  (1.5)  are  given  by 
(1.6)     u^  =  -C^x^^^^  -     \\il/2)  0-k^N-l; 


where  C,  and  H^  are  determined  by 

(1.7a)     \Cj^  =  D'(\+i  +  Ql.k+l>^ 

(l-^^>       \  =  Q2,k^°'(\+l  +  Ql,k+l>^ 

(1.7c)       F^  =  A'(F^^^  +  Q^^^^^)A  -  C^H^Cj^,    F^  =  0 

(1.8b)       \  =  (G^^k^l  +  B^^^)A  -  Z^Cj^,        B^  =  0, 

where  it  is  shown  in  the  dynamic  programming  solution  to  the  problem  that 

F,^  is  psd  and  therefore,  by  the  assumptions  on  0  ]. . -i  jQo  u  »  ^^^^   D»  ^  is 

nonsingular  and  C,  is  determined  uniquely  from  (1.7a). 

As  Kalman  has  shown,  there  is  a  close  relationship  between  equations 

(1.4)  and  (1.7)  which  allows  results  obtained  for  the  filter  equations,  (1.4), 

to  be  applied  to  the  optimization  equations  (1.7)  and  (1.8).   In  section  II 

of  this  paper  we  use  this  relationship  and  the  uniform  asymptotic  stability 

of  the  Kalman  filter  to  show  that  the  matrices  C  =  C  (N)  and  H~  Z' 

o    o         o   o 

H   (N)Z'(N)  in  (1.6)  converge  to  a  fixed  point  as  the  time  horizon  N  increas- 
es, when  the  costs  in  (1.5)  are  stationary.   This  result  is  extended  in  sec- 
tion III  to  the  case  of  discounted  costs.   In  section  IV  we  explore  the  im- 
plications of  these  results  for  the  aggregate  planning  problem  in  production 
scheduling.   Briefly,  these  results  imply  that  when  sales  are  generated  by  a 
linear  autoregressive  system,  then  the  first  period  decision  rules  approach 
a  fixed  point  as  the  time  horizon  is  increased.   Finally,  generalizations  to 
the  above  results  are  discussed  in  the  concluding  remarks. 

The  above  results  for  the  undiscounted  case  with  pure  quadratic  loss 
function  were  originally  proven  by  Kalman  [  5  ] ,  although  his  results  on  the 
"separation  theorem"  were  known  earlier  in  another  form  as  the  certainty 
equivalence  theorem,  see  Simon  [8].   Kalman's  results  were  applied  and  ex- 
tended to  the  problem  of  aggregate  planning  and  information  system  design 


I  i 


by  G.  B.  Kleindorfer  [6  1.   Anderson  et.  al.  extended  Kleindorfer's  work  in 
their  study  of  the  identification  problem  and  they  laid  most  of  the  ground- 
work for  the  analytical  methods  to  be  used  here.   The  present  work  extends 
past  results  on  the  convergence  of  the  first  period  decision  rules  to  the  case 
where  the  objective  function  contains  linear  terms  in  the  state  and  control 
variables  as  well  as  extending  known  results  to  the  discounted  case. 

II.   THE  UNDISCOimTED  CASE 

We  begin  by  studying  the  asymptotic  behavior  of  (1.4).   We  may  combine 
equations  (1.4)  to  obtain 

(2.1)  S^^^   =  A(Sj^  -  Sj^M'[MSj^M'  +  W]%Sj^)A'  +  V,  k>l. 
Moreover,  it  is  clear  from  (1.4)  that  the  matrices  {S^,S„,...}  uniquely 

determine  the  corresponding  sequences  {A  ,A  ,...}  and  {P^,P„,...}.  We  there- 
fore restrict  our  attention  to  a  study  of  (2,1)  and  note  the  following  result 
proven  in  Anderson  £t .  al^.  [  1  ]  . 

Theorem  1:  Let  M=I  and  let  V  be  pd  and  W  psd;  define  $on  the  set  T  of  pd  ma- 
trices by 

(2.2)  $(S)  =  AS(S  +  W)~"HjA'  +  V,        SeT 

so  that  S,=  "5(5,  ^),  k^2.   Then  $  has  a  unique  pd  fixed  point  S  ,  and 

<I>  (S)-*S  uniformly  on  T  as  n-**",  where  $  denotes  the  nth  iterate  of  $.  More- 
o         ^ 

over,    A,    ^  A      =   S    (S     +  W)~    ,    and  P     ->P     =S     -S(S     +  W)~   S      as  k-^^'. 
kooo  Roooo  o 

This  result  was  proven  originally  by  Kalman  [  5  ]  under  the  assumption  that 
the  system  (1.1)  and  (1.2)  is  completely  observable  and  com.pletely  controlla- 
ble.  His  proof  also  allows  for  M,  the  observation  matrix,  to  be  non-square. 


Results  of  a  similar  nature  are  also  contained  in  some  unpublished  research 
of  Professor  Lance  Taylor  of  Harvard  University. 


Theorem  1  may  also  be  easily  generalized  to  include  a  non-square  observation 

matrix. 

Corollary  1:   Let  V  and  W  be  pd  and  let  rank(M)  =  min(q,p);  define  <I>  on  the 

set  T  of  pd  matrices  by 

(2.3)  $(5)  =  A(S"-"-  +  M'w"-^l)"-'-A'  +  V,       S  eT. 

Then  for  any  S^  pd,  S,  =  'I>(S,_  ),  k^2.   Moreover,  $  has  a  unique  pd  fixed 

point,  S  ,  and   $  (S)  ^  S   uniformly  on  T  as  n  ^  <»,  where   $   denotes  the  nth 

iterate  of  $.   Furthermore,  h,    ^  ^     =  S  M' [MS  M'  +  W]~  ,  and  p,  ^-  p  = 

k    o    o     o         '      k    o 

S   -  S  M'[MS  M'  +  W]~HlS   as  k  ^  «>. 
o    o     o  o 

Proof:   We  first  verify  that  for  any  S   pd,  S   =  $(5  ),  k^2.   By  (2.1)  and 

the  fact  that  [MS^M'  +  W]t  =  [MS^^M'  +  W]"''"  since  W  is  pd,  we  only  need  to  show 
that 

(2.4)  (S"""^  +  M'W"-^!)"-'-  =  S  -  SM'[MSM'  +  Wj'HlS,        S  eT. 

To  demonstrate  the  validity  of  (2.4)  consider  the  following  calculations. 

(2.5)  (S""""  +  M'W"^M)(S  -  SM'[MSM'  +  W]""'"MS) 

=   I  -  M'[MSM'  +  W]~"SlS  +  M'W~"SlS  -  M'W"''>ISM'[MSM'  +  W]~"SlS 
=  I  +  M' (W""*"  -  [MSM'  +  W]"''"  -  w'^^ISMUMSM'  +  W]"''")MS 
However, 

(2.6)  w'HlSM' [MSM'  +  W]""""  =  W""'"(MSM'  +  W  -  W)[MSM'  +  W]"-"- 

=  W~"'"(I  -  W[MSM'  +  W]~^) 

=  W""""  -  [MSM*  +  W]""*-, 
so  that  substitution  of  (2.6)  into  (2.5)  yields  the  desired  result. 

Corollary  1  now  follows  from  theorem  1  since  when  M  is  of  full  rank,  M'W  "M 
is  invertible  and 

(2.7)  (S"-"-  +  M'V:""^!)"-'-  =  S[S  +  (M 'w'-'t^)  ""'"]"■'"  (M'w"  "Si)  "■'■ 

so  that  identifying  (M't/'^I)"  with  W  in  (2.2)  yields  the  desired  result. 

We  now  consider  the  relationship  between  the  control  equations  (1.7)  and 


the  Kalman  filter  equations  (1.4).   In  fact,  equations  (1.7)  may  be  combined 
to  yield 

(2.8)  \  =  ^'^\^i  +  <^i,^+l)^- 

^'^Vl  ^  Ql,k+l>°tQ2,k  +  °'^Vl  +  ^l,k+l)°]"'°'(\+l^l.k+l>^ 

^N  =  « 
or  letting  R,  =  F  +  Q    ,  k  =  1,2,...,N,  we  obtain 

(2.9)  R^  =  A'(R^^^  -  ViD[Q2^^  +  D ' R^^^D ] " ^D ' R^^^^ A  +  Q^^^.   k=l,...,N-l; 

Comparing  (2.9)  with  (2.1)  we  see  that  these  difference  equations  are  of 

precisely  the  same  form  if  we  identify  Ro  S,  A  <•+  A',  D  <->  M' ,  Q    <->  W, 

2  ,k 

^1  k^"*"  ^'  ^'^'^  ^~^'      '^"^'  ^^^^   ^1  k  "  *^1'  ^2  k  "  ^2  ^°^  ^^-^   ^'  ^^  ^^  clear 
that  a  study  of  the  asymptotic  behavior  of  F^  +  Q  =  R  =  R  (N)  as  N  ->-  oo  may 

be  obtained  from  corollary  1.   Indeed,  corollary  1  and  the  above  remarks  imply 

Theorem  2:  Let  rank(D)  =  min(p,r)  and  let  Q^  and  Q„  be  pd.   Define  y  on  the 

set  of  pd  matrices  T  by 

(2.10)  ^(R)  =  A'(R  -  RD[Q2  +  D'RD]""'"D'R)A  +  Q^,         ReT , 

N 
so  that  R,  =  >}'(R^^).   Then  m   has  a  unique  pd  fixed  point  R^  and  y  (R)  -)-  R^ 

N 
uniformly  on  T  as  N  ->-  <»,  where  y  denotes  the  Nth  iterate  of  m .      Moreover, 

H    (N)    =  Qo   +  D'4'^"-'-(QjD  ^  Q„   +  D'R.D  and  C    (N)    =    [Q.  +  D'>l'^"-'-(Q,  )D]"-'-D'4'^""'"(Q,  )A 

OZ  iZ"OZ  1  1 

->-   C^  =    [Q     +  D'R^D]~"''D'R^A  as   N  ->  «>. 

Now  let  us  consider  the  asymptotic  behavior  of  B^  =  B  (N)  in  (1.8)  which 

is  required  for  the  computation  of  Z  (N) ,  used  with  C  (N)  and  H  (N)  in  (1.6) 

o  o         o 

for  the  computation  of  the  first  period  controls,  u  .   From  (1.8)  we  obtain 

(2.11)  Bj^  =  \^^i^  -   DCj^)  +  G^^j^_^j^(A  -  DC^)  +  G2^j^Cj^,   k=l,2 , . .  .  ,N-1; 

^N  =  0 
In  order  to  show  that  B  =  B^  (N)  converges  to  a  fixed  point  as  N  ^-  <»,  we 

will  need  the  following  lemmas,  the  first  of  which  is  due  to  Stein  and  is  proven 


in  [  ].  ] . 

Lemma  1 ;   Let  Y  be  a  square  matrix  and  let  p(Y)  be  the  spectral  radius  of  Y. 

If  there  exists  a  pd  matrix  L  for  which  L  -  Y'LY  is  pd  then  p(Y)  <  1. 

Lemma  2 ;   Let  C^  =  [Q  +  D'R^D]   D'R^A,  where  R^  is  the  unique  pd  fixed  point 

of  4-.   Then,  p(A  -  DC^)  <  1. 

Proof;   By  definition  of  C^^,  we  have 

(2.12)  A  -  DC^  =  A  -  D[Q  +  D'R^D]~"'"D'R^A 

=  (I  -  D[Q2  +  D'R^D]"-'-D'R,.)A 
=  R;^(R^  -  R*D[Q2  +  D'R^D]"^D'R^)A 
Now  a  calculation  similar  to  the  proof  of  (2.4)  in  corollary  1  shows  that 

(2.13)  R^  -  R^D[Q2  +  D ' R^^D ] " ""-D ' R^  =  [R^"""  +  DQ'^D']'^ 
Therefore,  we  have  from  (2.12) 

(2.14)  A  -  DC^  =  R'^LR^"""  +  DQ2^D']'"'"A 

To  prove  the  assertion  it  will  suffice  by  lemma  1  to  exhibit  a  pd  matrix 
L  for  which  L  -  (A  -  DC^)'L(A  -  DC^^)  is  pd.  But  L  =  R^  is  such  a  matrix,  for 
by  definition,  R^  =  4'(R^) ,  and  therefore 

(2.15)  R^  -  (A  -  DC^)'R^(A  -  DC^)  =  A*  (R^  -  R^DlQ^   +  D'R^D]"-'d'R^)A  + 

Q^  -  (A  -  DC^)'R^(A  -  DC^) 
Now  using  (2.13)  and  (2.14)  in  (2.15)  we  obtain 

(2.16)  R^  -  (A  -  DC^)'R^(A  -  DC^)  =  A' (R^  -  R^D[Q2  +  D'R^D]"-'-D'R^)A  - 

A'[R;^  +  DQ2^D']"'^R^\r;;"^[R;^  +  DQ2^D']"^A  +  Q^ 
=  A'[R;^  +  DQ^^D'l'-^d  -  R;^[R;^  +  DQ2^D']""^)A  +  Q^ 

=  A'fR^-"-  +  DQ^-'-D' ]""""{ I  -  (R^"""  +  DQ'-'-D'  -  DQ2"'"D')[R^^  +  DO'^D' ]  ~''"}A  +  Q^ 
=  A^R^"*^  +  DQ2"'"D']~"''DQ2"'"D'[R^^  +  DQ^-'-D ']"''■  A  +  Q^  QED. 

We  now  show  that  B  (N)  converges  to  a  given  finite  vector,  B^.  For  con- 
venience, we  reverse  the  time  index  in  (2.11)  so  that  k  «^  N-k.  and 

(2.17)  B^^^  =  B^(A  -  DC^^^)  +  G^(A  -  DC^^^)  +  G^C^^^,    B^  =  0  . 


Theorem  3:   Let  B  be  defined  by  (2.17).   Then  B   converges  to  B.  given  by 

t      ■  t  * 

(2.18)  B^  =  [G^(A  -  DC^)  +  G^C^ILI  -  A  +  DC^]"-"" 

where  C.  is  given  in  lemma  2.   Consequently,  B  (N)  converges  to  B.  as  N  ->  ». 

Proof:   Define  the  matrices  5  and  Q.     by 

t      t   -^ 

(2.19)  E^  =  A  -  DC^^^ 

(2.20)  Q^   =  G^(A  -  DC^^j^)  +  G2Cj.^^ 

so  that  (2.17)  becomes 

(2.21)  B^^^  =  B^E^  +  ^^  ,        B^  =  0  . 

Now  since  C   ->-C^,  5  ->-E^  =  A-  DC  ,  and  by  lemma  2,  p(E^)  <  1,  so  that 

(2.21)  is  asymptotically  stable.   From  (2.21)  we  obtain 

(2.22)  B^^,  =  B  n^   =,  +   z5  a  (nj  ,_Li5.) 

t+1    o  j=o  j       k=o  \     j=k+l  J 

We  now  remark  that  since  p(H^)  <  1,  I  -  E^  is  invertible,  and 

(2.23)  (I  -  E^-^  =   Z^^^E^   , 

so  that  letting  B  =  Z:^^   fi^(n^^j^_^^E^)  .where  Q^   =  G^(A  -  DC^)  +  G2C^,  we  have 

A  it 

B  ->  B..   Thus  it  suffices  to  show  that  Ib  -  B  |->  0,  where  I  •  I  is  the  Euclidi- 
t    "  t    t 

an  norm.   To  show  this,  we  note  that  p(E^)  <  1  implies  (see  Varga  [10],  p.  67) 
the  existence  of  an  integer  r^l  for  which  \bA    <1,  and  since  E  ->•  E^,  there 
exist  p   <  1  and  an  integer  k  >  1  for  which 

(2.24)  Max  [  1  n^^J  E .  |  ,  |E^|]  <  p^  ,  k>k^. 

Since  Q,    is  a  convergent  sequence,  there  exists  a  uniform  upper  bound,  U^ ,  such 
that  |fi,  I^U  for  all  k.   Moreover,  (2.24)  implies  for  s^l,  and  k^k   that 

(2.25)  Max  l\jl]__^^^   E  .  |  ,  |  Hr""' 1 1^  P^Max  [  |  nj^,^,^^^  E^  |  ,  |  Er""'- |  ] 

In  particular,    £im        llT.    .    E.l=  0   for  every   j   ^0,   so   that 

o  . 

(2.26)  ^im^        sup    Ib^^,    -  B^^J  ^  «,im^        supU^df      |n^,^iS.|   +    |n^,_^,E*l} 

t->«'  '    t+1  t+1'  t^<»  1     k=o'    j=k+l  j'         '    j=k+l   *' 

=  4im^_  sup   U,{z5    .|n^  T,.iH.|    +    IH^"^"""-!} 
t^oo        ^      1     k=j  '    j=k+l  J  '         '    *  ' 


-so-!  s   ,,    ,_   t    I    t  -    I    J.      I  _t-k-l-rs  I  , 

t-Ko        ^   '^o      1     k=j      j=k+l+rs      j'  '    *  '      -"o        o 


o 


But  (2.25)  implies  for  i  >  k  and  for  iS 0 
^         -^o   o 

t     t  Po   Jo  +i+r   3o+i+k 

(2.27)  E    I  n     5.|    ^    E   {   Z       I    n    E.jlpP 
k=j    j=k+i   ^         p=o    k=j  +i    i=j  +i   ■'    ° 

where  p  is  the  largest  integer  less  than  or  equal  to  (t-j  -i)/r.   Therefore, 

(2.28)  lim^_  sup  E^^  .  I  nj^^^^  H  .  |  -  U^/d-P^) 
Then  (2.26)  and  (2.28)  imply 

(2.29)  iim^_  sup  |b^_^^  -  B*^J  ^  2p^  U^U2/(l-p^) 

which  may  be  made  arbitrarily  small  by  proper  choice  of  s.   This  completes  the 
proof  of  theorem  2. 

We  may  summarize  the  results  of  this  section  as  follov7s.   Let  u  (N)  be  the 
optimal  first  period  controls  given  by  (1.6)  as 

(2.30)  u  (N)  =  -C  (N)x  I   -  1/2  h"-'-(N)z' (N)  . 

o        o    o  I  o        o     o 

Let  D  be  of  full  rank  and  let  Q,  and  Q„  be  pd.   Then  u  (N)  converges  to 

L  A  o 

(2.31)       u^  =  -c^x^i^  -  1/2  h;^  z; 

as  N  ->  ».   The  values  of  the  parameters  in  (2.31)  are  given  below  and  x  i   is 

o|o 

the  linear  least  squares  estimate  of  the  system  at  the  present  time. 

(2.32)  R^  =  A'(R^  -  R*D[Q2  +  D' R^D]"-'-D'R^)A  +  Q^ 

(2.33)  C^  =  [Q2  +  D'R^D]"-'"D'R^A 

(2. 34)  H^  =  Q^  =  D'R^D 

(2.35)  Z^  =  G^D  +  G2  +  B^D 

(2.36)  B^  =  [G^(A  -  DC^)  +  G^C^]!!  -  A  +  DC^]"''- 

It  should  be  remarked  that  in  practice  one  would  determine  R^  by  iteration 
of  (2.9)  with  k  ^   N-k  until  successive  values  of  R,  and  R.  ,-,  were  within  a  de- 
sired tolerance.   In  this  process  it  should  be  recalled  that  convergence  of  R, 
to  R^  is  uniform.   In  fact,  it  can  easily  be  shown  on  the  basis  of  the  proof  of 
the  above  theorem  1  in  Anderson  et.  al.  [1],  that  (with  k  ♦>  N-k) 
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(2.37)     R^  =  4'^(Q3^):£  f^CA'  [DQ^^D'  ]"^A  +  Q^):^A'  [DQ^^D'  ]  ^A  +  Q^ 

where  X^Y  in  (2.37)  if  Y  -  X  is  psd.   Thus,  since  both  ^^(Q.)  and 

N      -1   -1 
^  (A' [DQ„  D' ]   A  +  Q^)  are  converging  to  the  fixed  point  R^,  one  can  use  the 

(some  convenient)  norm  of  the  difference  of  these  two  quantities  to  obtain  a 

I  N  I 

precise  bound  on  If  (Q. )  -  R. | . 


III.   THE  DISCOUNTED  CASE 

Let  a  be  a  given  discount  factor,  0<a<l,  and  consider  the  minimization 
(1.5)  with 

^^•^^     ^i,k  "  ''''  ^i'  ^i,k  =  "^  ^i'   i=1.2;  k>0. 
In  this  case  equations  (1.7)  and  (1.8)  yield 

(3.2)  R^  =  A'(Rj^_^^  -  Rj^_^^D[D'R^_^^D  +  a^Q2]'-^D'R^_^j^)A  +  a^Q^ ,   1-k^N-l; 

k+1  k 

(3.3)  Bj^  =  Bj^_|_^(A  -  DCj^)  +  a   ■'g^(A  -  DC^^)  +  a''G2Cj^  ,  B^^  =  0,  O^k^N-1; 

where  C,  is  determined  by  (1.7)  and  where 

(3.4)  \  =  \  +  ^''Qi  '  ^j  =  ""Qi- 

We  now  show  that,  by  redefining  the  system  parameters,  (3.2)  can  be  put 

in  the  form  of  (2.9)  with  stationary  parameters,  so  that  the  desired  asymptotic 
properties  follow  from   theorem  2.   We  begin  by  defining  the  parameters 

(3.5)  A  =  BA,   ^^  =  Q^,   D  =  D,  ^^  =  (l/a)Q2,   3^  =  a. 

-k 
Then  letting  R^  =  °'  R,  ,  we  have  from  (3.2) 

(3.6)  4  =  A'(a-\^^  -  «-V,D[D'R^^^D  +  a\]-^D'R^^^)A  +  Q^ 

=  (eA)'(a~''"\^^  -  a"^"\^^D[D'(a"''"\^^)I>+a"\]"^D'a"''"\_^^)(BA) 

or,  using  (3.5), 

(3.7)  R^   =  A'  (R^^^  -4^i^tD'R,^^D  +  ^^fh^^^^U   +  S, 

Thus,  (3.7)  is  precisely  of  the  same  form  as  (2.9)  and  theorem  2  therefore 
implies  that  R,  (N)  =  aR,  (N)  converges  uniformly  to  a  fixed  point,oiR^,  as  N  ^  «>. 
This  also  implies  the  convergence  of  C  (N)  and  H  (N)  to  C.  =  H.  D'ctR.A  and 
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H^  =  Q-  +  D'  R^D,  respectively. 

Similarly,  the  equation  (2.11)  for  B  becomes  for  the  discounted  case 

k+1  k 

(3.8)  Bj^  =  Bj^^^(A  -  DC^)   +    a       G^(A  -  DC^^)  +  a  G2C^,   B^^  =  0. 

-k 
Let  \  =    a.     B  .   Then  (3.8)  implies 

(3.9)  B^  =  \+l"^^  "  "^S^  "''  ^iCt^A  -  DCj^)  +  G^C^    . 

If  we  identify  E,    =  a(A  -  DC  )  and  ft  =  G  a(A  -  DC  )  +  G  C  ,  then  we  may  pro- 
ceed as  in  theorem  3  to  prove  the  convergence  of  B^^  (N)  to  a  finite  vector  B^^^ 
provided  that  p(a[A  -  DC^])  <  1,  where  C^  is  the  fixed  point  of  C  (N)  defined 
above.   To  show  this  we  note  that 

(3.10)  C^  =  [Q^  +  D'aR^D]"-'-D'aR^A 

=  [a~"'"Q2  +  D'R^D]"-'-D'R^A 

=  6[^2  "^  5.'R*R3~V^*^ 

and  therefore,  using  (3.5),  we  obtain 

(3.11)  a(A  -  DC^)  =  6(A  -  DC^) 

where  C^^  is  the  fixed  point  of  the  stationary  system  with  parameters.  A,  £,  ^  , 

and  0„,  corresponding  to  C  (N) .   But  p(e[A  -  DC.])  =  6p(A  -  DC.)  and 
Z  — o  —   — **       —   — ^ 

p(A  -  DC^)  <  1  by  lemma  2,  so  that  the  desired  result  follows  as  in  the  proof 
of  theorem  3.   We  may  summarize  the  results  of  this  section  in  the  following  man- 
ner. 

Theorem  4:   Let  u  (N)  be  the  optimal  first  period  controls  given  by 

(3.12)  u  (N)  =  -C  (N)x  I   -  1/2  h"-^(N)Z'(N)  . 

o        o    o  I  o        o     o 

Let  D  be  of  full  rank  and  let  Q,  and  Q_  be  pd.   Then  u  (N)  converges  to  u.  given 

i       z  o  " 

by 

(3.13)  u^  =  -C^x  I   -  1/2  h"^! 


o  o 


*  "* 


as  N  ^  «>.   In  (3.13)  x  i   is  the  linear  least  squares  estimate  of  the  system  state 
at  the  present  time  and  the  parameters  C^,  H^,  and  Z^  are  determined  by 
(3.14)    R^  =  A'(R^  -  R^[Q2  +  D'R^]"-'-D'R^)A  +  ^^ 
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(3.15)  C^  =  [Q2  +  D'aR^D]~-'"D'aR^A 

(3.16)  H^  =  Q2  +  D'aR^D 

(3.17)  Z^  =  aG^D  +  G^  +  aB^D 

(3.18)  B^  =  [G^a(A  -  DC^)  +  G2C^][I  -  a(A  -  DC^)]"-*- 
where  A,D,  ^^  ,  and  ^^  ^'^^  given  in  (3.5). 


IV.   APPLICATIONS  AND  EXTENSIONS 

Although  the  results  of  the  preceding  sections  have  wider  applicability, 
we  restrict  ourselves  to  a  brief  exploration  of  their  implications  for  the  ag- 
gregate planning  of  production  and  work  force  (see  Holt  et^.  al.  [2]).   This 
discussion  will  serve  to  highlight  as  well  the  limitations  of  the  above  analysis, 

Following  Holt  et^.  aj^.  [  2  ]  ,  we  first  assume  the  following  model  for  the 
aggregate  planning  problem. 
(4.1)      mnE[Ll^^f^a^^^,V^,^^^,U^)} 
subject  to 

<^-2)     \+l  -\^\-\ 

^^•3)       Vi  =  \^\ 

(4.4)     I  ,  W  given, 

where  f  is  a  quadratic-linear  cost  function  in  its  arguments  and  represents 

ilv 

period  k  costs.   I,  is  the  inventory  at  the  beginning  of  period  k,  P  is  the 
aggregate  production  in  period  k,  S,  is  the  sales  in  period  k,  W  is  the  work 
force  at  the  beginning  of  period  k,  and  U,  is  the  change  in  work  force  during 
period  k. 

In  order  to  reduce  the  above  problem  to  the  form  of  (1.1)  and  (1.2),  we 
must  assume  that  the  sales  are  generated  by  a  first  order  autoregressive  scheme 
of  the  form 


(4.6)     A^    =>^\"k 
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where  ?/  =  (S,  ,  S    ,  ...,  S,_  ).   Then  (4.5)  and  (4.6)  could  be  incorporated 
into  (4.2)  to  yield  a  system  of  the  form  of  (1.1)  and  (1.2).   Besides  only  be- 
ing able  to  consider  sales  generated  by  an  autoregressive  scheme  ,  the  present 
results  are  also  limited  to  costs  which  are  separable  in  the  state  and  control 

variables.   This  would  rule  out,  for  example,  costs  of  the  form  (see  [  2]) 

2 
Cost  of  Overtime,  =  c^ (P,  -  c„W  )   +  c_P  +  c.W  ,   c^ ,  c„>  0, 

since  such  costs  lead  to  terms  of  the  form  -2c,c„P,  W,  .   For  the  above  reasons 

1  2  k  k 

it  seems  appropriate  to  generalize  the  fundamental  model  (1.1) -(1.2)  to  the  form 

(4.7)  x^^^  =  Axj^  +  Du^  +  Sj^  +  Vj^ 

(4.8)  y^   =  Mx^  +  w^ 

where  all  quantities  above  are  defined  as  in  (1.1)-(1.2)  except  s,  which  is  a 
deterministic  p-vector. 

Recent  work  (see  [9])  in  Kalman  filter  techniques  has  generalized  the 
underlying  model  to  which  the  Kalman  filter  is  applicable  to  the  form  of  (4.7)- 
(4.8).   The  fundamental  filter  equations  (1.4)  remain  unchanged  in  this  case 
and  therefore  the  results  on  their  asymptotic  behavior  are  still  applicable. 
Moreover,  (4.7)  and  (4.8)  are  clearly  directly  related  to  the  form  of  the  aggre- 
gate planning  problem  (4.1)-(4.4).   It  remains  to  be  determined  whether  the 
essential  properties  of  equations  (1.6)-(1.8)  will  hold  for  the  system  (4.7)- 
(4.8).   It  is  my  conjecture  that  the  results  of  theorems  2, 3, and  4  hold  for  the 
system  (4.7)-(4.8)  whenever  the  cost  function  in  (1.5)  is  a  quadratic-linear 
function  (including  state-control  cross-product  terms)  provided  that  the  cost 
function  is  convex  and  strictly  convex  in  the  controls,  u,  ,  and  when,  in  addi- 
tion, the  terms,  s,  ,  are  bounded  by  a  stationary  linear  system.   Verification 


Hi 


of  this  conjecture  involves  first  resolving  the  dynamic  program  leading  to 
(1.6)-(1.8)  with  added  cross  product  terms  in  u^  and  x^^  and  subject  to  (4.7) 
and  (4.8)  instead  of  (1.1)  and  (1.2).   The  present  work  and  that  reported  in 
[  1]  and  [  7  ]  provides  a  foundation  for  further  studies  in  this  direction. 
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